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ABSTRACT 

Due  to  increased  use  of  imaging  sensors  in  military  aircraft,  future  combat  airplane  pilots  will  need  onboard  artificial 
intelligence  for  aiding  them  in  image  interpretation  and  target  designation. 

This  document  presents  a  system  which  is  able  to  create  high-resolution  artificial  SAR  imagery.  The  resulting  images 
can  be  used  to  facilitate  automatic  target  recognition  (ATR)  algorithm  development.  The  system  provides  an  interface 
that  allows  dynamically  requesting  imagery  depending  on  the  location  and  heading  of  a  simulated  carrier  platform. 
Landscapes,  structures  and  target  signatures  are  generated  based  on  digital  terrain  elevation  and  cultural  data  and  3d 
models.  Post-processing  algorithms  for  overcoming  weaknesses  of  digital  terrain  databases  and  improving  image 
realism  are  presented. 

Simulated  sensor  imagery  is  useful  in  a  wide  range  of  applications,  two  of  which  are  training  of  ATR  algorithms  and 
sensor  simulation  in  flight  simulation  environments. 

Using  an  existing  ATR  method  as  an  example,  the  applicability  and  the  influences  of  synthetic  imagery  on  ATR 
training  are  shown  and  first  approaches  how  to  validate  the  correctness  of  the  imagery  are  outlined.  The  integration  of 
the  system  into  a  flight  simulator  in  the  context  of  interfacing  and  control  topics  is  presented  as  another  example  for  its 
applicability.  As  an  outlook,  an  example  of  using  external  databases  for  creating  imagery  of  crisis  areas  is  outlined. 
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1.  INTRODUCTION 


1.1.  SAR  image  interpretation 

Currently  most  military  SAR  sensors  are  installed  in  reconnaissance  satellites  or  airplanes.  These  sensors  obtain  high- 
resolution  imagery  of  a  target  area  and  transfer  it  to  ground  stations  where  specially  trained  professional  image 
interpreters  scan  these  data  for  man-made  objects.  The  results  are  then  used  to  plan  and  execute  precision  attack 
missions.  After  the  attack,  damage  assessment  can  be  done  on  newly  gathered  imagery. 

Future  combat  aircraft  are  expected  to  display  a  self-targeting  capability  which  denotes  the  ability  to  run  through  the 
entire  chain  of  sensor  deployment,  image  interpretation,  target  designation,  mission  planning  and  execution  all  on  their 
own.  This  puts  a  lot  of  additional  workload  on  the  pilot  who  would  have  to  scan  huge  images  for  potential  targets 
while  flying  through  or  close  to  enemy  territory. 

Another  important  issue  is  that  SAR  imagery  looks  quite  different  compared  to  the  classical  optical  images  that  pilots 
are  used  to.  In  order  to  reduce  the  workload  on  the  pilot  it  is  necessary  to  aid  him  by  means  of  automatic  image 
interpretation  [  1  ]  [2] . 

1.2.  Artificial  SAR  imagery  for  SAR  ATR 

We  have  developed  a  training-based  method  for  automatic  target  recognition  (ATR)  which  can  be  used  for  visual, 
infra-red  as  well  as  SAR  imagery.  This  ATR  system  is  an  ensemble  of  neural  networks  with  Gaussian  and  Laplacian 
image  pyramids  as  pre-processing  steps.  It  is  described  in  detail  in  [3]. 
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Because  of  its  learning-based  approach,  our  method  requires  the  existence  of  realistic  and  diverse  imagery  with 
detailed  knowledge  about  the  objects  depicted  in  the  image.  In  SAR  imagery,  object  signatures  change  drastically 
depending  on  their  orientation  to  the  radar  beam.  They  are  also  changed  by  pixel  correlation  and  speckle  effects. 

In  order  to  counter  these  problems,  the  ATR  method  must  be  shown  many  pictures  with  different  viewing  geometries. 
It  is  costly  and  time-consuming  to  gather  real  SAR  imagery  that  meets  all  these  preconditions.  This  was  the  driving 
force  for  the  development  of  the  SAR  simulator  at  the  military  aircraft  business  unit  at  EADS. 

2.  THE  SAR  SIMULATOR  AT  EADS 


2.1.  System  requirements 

The  goals  mentioned  above  impose  several  design  requirements  on  the  system.  Of  course  it  must  allow  for  the  realistic 
simulation  of  landscapes  and  man-made  objects,  an  exact  definition  of  which  is  still  to  be  developed  (see  paragraph  3). 
For  the  simulation  of  different  sensors  and  processing  parameters,  the  basic  simulation  parameters  like  bandwidth, 
sampling  frequency  and  polarisation  must  be  easily  configurable.  In  order  to  obtain  simulated  target  signatures  which 
are  very  close  to  reality,  it  is  necessary  to  simulate  SAR  specific  effects  like  noise,  speckle,  foreshortening,  layover, 
comer  reflector  effect  and  edge  currents. 

The  goal  of  integrating  the  SAR  simulation  into  a  flight  simulator  environment  introduces  further  requirements  to  the 
simulation  like  short  processing  time  (order  of  seconds  to  minutes),  connection  to  a  terrain  database  and  accessibility 
of  the  provided  services  via  a  network  . 


2.2.  Basic  components 


image 

generator 


Figure  1  Basic  components 

The  SAR  simulator  consists  of  three  main  modules:  image  generator,  control  script  and  database  control.  The  image 
generator  module  is  responsible  for  the  generation  of  SAR  imagery  from  the  input  data.  These  data  are  created  by  the 
database  control  module,  which  extracts  them  from  terrain  and  target  database  according  to  the  request  that  was 
received  by  the  control  script.  As  the  name  suggests,  the  control  script  governs  the  data  exchange  between  image 
generator  and  the  database  control.  But  it  also  provides  a  network  interface  for  collaboration  with  external  software 
and  a  user  interface  for  accessing  sensor  and  processing  parameters.  Figure  1  illustrates  the  collaboration  of  these  basic 
modules.  The  arrows  indicate  data  and  control  flow. 

2.3.  System  implementation 

The  basis  of  the  SAR  simulator  is  an  EADS  proprietary  radar  image  generator  called  SAR-View.  It  was  originally 
developed  at  the  DaimlerChrysler  research  center  in  Ulm  and  is  still  being  expanded  and  developed  further.  The  basic 
steps  that  lead  to  the  synthesis  of  an  image  using  this  software  are  depicted  in  figure  2: 

After  a  set  of  terrain  data  has  been  extracted  from  a  terrain  database,  a  landscape  module  transforms  and  interpolates 
the  terrain  according  to  the  required  resolution  of  the  synthetic  image  and  the  viewing  geometry  (platform  heading). 
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The  landscape  module  uses  a  map  of  height  values  to  describe  the  terrain  surface  and  a  map  of  feature  indices  to 
denote  terrain  cover  like  paved  road,  forest  or  water.  From  the  orientation  of  surface  elements  and  their  material  codes 
radiometric  expectation  values  can  be  inferred  for  ground  and  slant  ranges.  The  target  module  relies  on  3d  CAD 
models  of  man-made  objects  and  uses  a  combination  of  geometric  and  physical  optics  to  calculate  radar  cross  sections 
(RCS)  for  the  simulated  targets. 

For  the  generation  of  realistic  target  signatures  the  3d  models  must  be  very  detailed  and  accurate  (at  least  some 
thousand  surfaces).  After  the  RCS  calculation  for  the  targets  is  finished,  the  result  is  integrated  with  the  landscape. 
Then  pixel  correlation  and  speckle  effects  are  computed  to  give  the  synthetic  SAR  image. 

radiometric 
feature  map  expectation 


synthetic  SAR  image 


Figure  2  The  process  of  image  formation  in  the  SAR  image  generator 

The  SAR  simulator  also  contains  a  database  control  module.  Its  graphical  user  interface  is  illustrated  by  figure  4.  This 
module  supports  access  to  the  terrain  database  and  provides  terrain  and  target  data  in  a  format  that  can  be  processed  by 
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Figure  4  The  database  access  module 


Figure  3  Data  flow  within  the  SAR  simulator 


the  image  generator.  Currently  DTED  /  DFAD  level  1  data  are  used  for  height  and  feature  data.  In  the  flight  simulator 
environment,  this  module  calculates  the  irradiated  surface  area  from  the  current  aircraft  position  and  sensor  alignment. 
After  that,  a  height  and  feature  map  for  the  calculated  area  are  extracted  from  the  terrain  database.  It  also  calculates  if 
any  targets  contained  in  the  target  database  are  within  the  calculated  surface  area.  If  there  are  any  targets  in  the  scene, 
their  local  coordinates  are  computed.  In  standalone  mode,  this  module  allows  for  the  placement  of  targets  within  the 
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terrain  database.  The  menus  for  this  mode  are  shown  in  figure  4.  The  target  location  data  are  kept  in  a  separate  text  file 
for  scenario  management.  This  mechanism  allows  other  objects  like  houses  or  power  line  poles  to  be  added  to  the 
database. 

The  control  script  is  the  superior  instance  that  controls  all  activities  within  the  SAR  simulation  software  and  the 
communication  with  external  programs.  It  is  written  in  ExpectK  which  is  a  superset  of  Tel,  TK  and  Expect 
functionality.  An  overview  of  the  data  flow  between  this  module  and  its  subordinated  programs  is  shown  in  figure  3.  It 
handles  all  simulation  parameters,  user  interaction  and  network  requests. 

The  script  provides  a  graphical  user  interface  (GUI)  for  convenient  modification  of  simulation  parameters  at  run-time. 
The  GUI  is  shown  in  figure  5.  Because  of  their  large  number,  all  controls  of  the  GUI  can  not  be  explained  here,  but  it 
allows  adjusting  sensor  parameters  (e.g.  chirp  bandwidth,  carrier  frequency,  depression  angle)  and  processing 
parameters  (hamming  weighting,  brightness).  It  displays  target  positions  within  the  scene,  shows  a  thumbnail  version 
of  the  generated  image  and  allows  the  user  to  select  the  desired  output  data. 

If  a  request  for  an  image  is  received  on  the  network  socket  (TCP),  the  control  script  provides  the  database  access 
module  with  the  data  needed  for  the  extraction  of  the  chosen  terrain  area.  This  communication  uses  standard  unix  I/O 
channels  (stdin/stdout).  The  database  access  module  then  computes  the  target  area  and  generates  files  containing  the 
elevation  and  feature  maps  as  well  as  text  files  containing  target  data  if  there  are  any  in  the  scene. 
These  files  are  used  by  the  SAR-View  /  SAR-Target  modules  (image  generator)  to  form  the  imagery  according  to  the 
adjusted  parameters  and  proceeding  through  the  steps  explained  in  figure  2.  The  generated  image  is  then  sent  to  the 
requesting  instance  via  a  TCP  network  socket.  For  ATR  evaluation,  ground  truth  data  can  be  transferred  along  with  the 
image  via  a  UDP  socket.  The  SAR  simulation  computer  is  a  Linux  based  Pentium  IV  PC  driven  by  two  2.4  GHz  CPUs. 
After  successful  integration  it  was  possible  to  generate  SAR  pictures  of  any  area  in  southern  Germany  (restricted  by  the 
size  of  the  terrain  database).  An  example  is  shown  in  figure  6.  This  image  has  a  size  of  about  1000x1000  pixels.  It 
covers  an  area  of  1500m  x  1500m  and  was  made  using  a  simulated  sensor  depression  angle  of  20  degrees.  The 
quadratic  blocks  are  flat  buildings  that  have  been  modeled  into  the  landscape.  The  time  needed  to  compute  this  image 
on  the  hardware  described  above  was  about  10  seconds. 

Using  the  network  interface  of  the  control  script  it  is  possible  to  simulate  SAR  image  for  any  position  within  the  terrain 
database  which  is  curently  restricted  to  southern  Germany  but  will  be  expanded  soon.  A  request  will  contain  the 
position  and  orientation  of  the  sensor  platform  and  the  alignment  of  the  sensor  on  the  platform  itself.  From  these  data 
the  simulator  will  determine  the  viewing  area  and  generate  an  image. 

2.4.Terrain  databases  and  post-processing 

The  DFAD  /  DTED  data  structure  has  some  drawbacks  for  the  creation  of  realistic  imagery  as  it  models  only  surface 
height  and  cover.  Moreover  there  is  no  connection  between  these  properties.  DTED  data  do  not  model  elevation  with 


Figure  5  GUI  of  the  SAR  simulator 


Figure  6  Sample  image 


respect  to  surface  cover,  which  means  that  there  will  be  no  height  difference  between  a  forest  and  a  bordering  meadow. 
For  man-made  objects,  DFAD  only  provides  technical  data  (e.g.  diameter,  height)  but  no  models. 
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Our  SAR  simulator  overcomes  these  drawbacks  by  inserting  a  post-processing  step  after  the  creation  of  height  and 
feature  maps  for  the  viewing  area  is  completed.  The  feature  map  is  analysed  and  the  height  map  gets  processed 
according  to  the  knowledge  gained  in  the  feature  map  analysis.  For  example:  in  forest  areas  all  surface  points  are  raised 
by  a  random  value  representing  the  height  of  tree  tops.  Road  are  lifted  above  their  surroundings  by  half  a  meter.  The 
following  pictures  (figures  7  and  8)  illustrate  the  effect  of  landscape  post-processing  on  the  resulting  image.  The  bright 
area  in  figure  7  is  covered  by  forest,  the  quadrilateral  structures  result  from  the  interpolation  of  the  underlying  height 
model.  In  figure  8  this  forest  area  looks  more  3 -dimensional  due  to  the  height  modulation  that  has  been  done.  The  edge 
of  the  forest  directed  towards  the  radar  sensor  lights  up  brightly  like  a  real  forest  would  and  there  is  a  shadow  area  on 
the  right  side  of  it.  In  the  enlarged  areas  in  figure  8  the  upper  rim  of  the  road  lights  up  compared  to  figure  7. 

In  the  case  of  high  resolution  imagery  (1.5m  or  better),  single  trees  become  visible  in  the  picture.  In  this  case  cones  of 
random  height  can  be  inserted  at  random  locations  within  forested  areas.  This  kind  of  processing  is  shown  in  figure  9. 


Figure  7  SAR  image  without  landscape  Figure  8  SAR  image  with  landscape  post-  Figure  9  Simulation  of  single  trees  for 
post-processing  processing  very  high  resolutions 


The  images  depicted  in  figures  7  through  13  were  created  using  a  depression  angle  of  about  8  degrees  in  a  side-looking 
geometry  .  The  carrier  signal  frequency  is  about  10  Ghz.  The  carrier  platform  was  flying  at  a  height  of  about  10  000  m 
at  a  speed  of  about  250  m/s. 

The  quadrilateral  structures  in  figure  7  are  visible  not  only  in  the  forested  area  but  also  on  the  meadow  areas  above. 
These  unrealistic  effects  can  be  overcome  by  using  a  better  height  interpolation.  Figure  10  shows  an  image  using  linear 
interpolation  while  figure  1 1  shows  a  landscape  in  SAR  which  has  been  interpolated  using  spline  curves.  Shadows 
which  show  up  as  dark  quadrilaterals  or  triangles  in  figure  10  look  rounded  in  figure  11.  The  layover  effect  that  slopes 
facing  the  sensor  produce  can  be  seen  as  bright  patches  around  the  group  of  buildings  in  figure  1 1  only  . 

While  simulating  high  resolution  imagery  using  current  terrain  databases  there  is  a  good  chance  of  viewing  a 
homogenous  area  like  a  crop  field.  This  problem  can  be  overcome  by  using  height  variations  that  are  typical  for 
different  crops.  Maize  will  have  a  different  height  profile  to  wheat.  These  height  variations  can  be  modeled  directly 
into  the  landscape  or  be  used  as  parameters  for  determining  the  corresponding  RCS  function.  Using  this  approach  a 
homogenous  surface  can  be  subdivided  into  different  regions  thus  looking  more  realistic. 
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Figure  10  Linear  interpolation  Figure  11  Spline  interpolation 


DFAD  contains  only  the  type  and  dimensions  of  man-made  objects  which  is  not  enough  for  generating  images  of  such 
artificial  objects.  To  resolve  this,  it  is  possible  to  extract  the  positions  of  power  poles,  bridges  and  similar  structures 
from  DFAD.  In  our  simulator  these  data  are  stored  in  a  text  file  and,  in  case  one  is  located  in  the  viewing  area,  an 
appropriate  CAD  model  is  inserted  into  the  scene.  The  following  pictures  (figures  12  and  13)  show  the  effect  of 
replacing  a  simple  height  map  modulation  (using  the  height  information  contained  in  DFAD)  with  a  detailed  CAD 
model  of  a  power  pole.  Figure  12  shows  a  3d  model  for  a  power  pole  (left  side)  compared  to  the  result  for  a  landscape 
height  modulation  using  DFAD  data  (right  side).  Figure  13  shows  a  SAR  image  with  these  two  objects  placed  into  a 
landscape.  Using  the  landscape  modulation  results  in  a  single  bright  patch  and  a  solid  shadow  (right  side).  The  3d 
model  for  the  power  pole  has  been  transformed  into  a  finely  structured  SAR  signature,  with  a  realistic  layover  effect 
(pole  looks  like  it's  bent  toward  the  sensor)  and  a  detailed  shadow. 


Figure  12  CAD  model  of  a  power  pole 
(left)  and  power  pole  as  height  map 
modulation  (right) 


Figure  13  Synthetic  SAR  image  using 
CAD  model  (left)  and  height  modulation 
(right) 


The  realism  of  the  generated  images  also  depends  on  the  quality  and  detail  of  the  used  terrain  database.  In  order  to 
improve  the  simulation  quality  it  is  useful  to  evaluate  other  databases  as  well.  For  Germany,  an  evaluation  of  the 
ATKIS  and  ALK  data  which  are  much  more  detailed  than  DTED  /  DFAD  level  1  is  planned.  Both  ATKIS  and  ALK 
data  are  constrained  to  Germany  and  stringent  access  restrictions  apply.  The  height  data  that  have  been  acquired  by  the 
SRTM  space  shuttle  mission  don't  have  the  required  precision.  Our  future  goal  is  to  use  VMAP  data  for  the  simulation 
which  is  an  upgraded  and  more  detailed  version  of  the  DTED  /  DFAD  database.  The  following  pictures  (figures  14  and 
1 5)  illustrate  the  different  degree  of  detail  of  DFAD  and  ATKIS  for  a  typical  area  of  Germany.  ATKIS  contains  more 
roads,  settlements  and  surface  structure  as  is  to  be  seen  in  figure  15. 
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Figure  14  DFAD  sample, 
2000x2000m 


Figure  15  ATKIS  sample, 
2000x2000m  for  the  same  area  as 


figure  14 


3.  VALIDATION 


The  appearance  and  quality  of  SAR  imagery  is  influenced  by  many  simulation  parameters  (e.g.  chirp  bandwidth, 
carrier  frequency,  signal-to-noise  ratio),  the  realism  and  detail  of  CAD  models  and  the  detail  of  the  terrain  database. 
Because  of  these  facts  the  imagery  generated  by  a  simulator  does  not  necessarily  look  like  imagery  from  real  sensors. 
In  order  to  be  able  to  develop  ATR  algorithms  on  the  basis  of  synthetic  images  that  also  work  on  real  images,  it  must 
be  assured  that  the  synthetic  images  look  as  similar  to  the  real  ones  as  possible.  For  this  reason  it  is  necessary  to 
develop  methods  which  allow  for  objective  comparisons  between  real  and  synthetic  pictures  of  given  scenes.  In  figure 
16  synthetic  imagery  based  on  different  terrain  data  are  compared.  The  image  rendered  on  the  basis  of  DTED  data  (left) 
shows  less  detail  than  the  ATKIS-based  image  (right). 


Figure  16  Comparison  of  synthetic  SAR  imagery  of  the  same  landscape  area  using 
DFAD/DTED  data  (left)  and  ATKIS  data  (right) 


A  classical  approach  for  validating  imagery  is  statistical  analysis  of  image  properties.  Histogram  evaluation  can  help  to 
assure  that  typical  image  properties  like  its  gray  value  distribution  are  realistic.  These  distributions  vary  in  a  typical 
way  depending  on  the  processing  parameters  of  the  image.  There  are  property  measures  which  can  be  employed  for 
comparing  images  like  the  „co-occurrence  matrix^  [4].  The  co-occurrence  matrix  determines  a  value  for  the  gray  value 
arrangement  in  an  image.  It  is  the  basis  of  many  image  property  measures. 

It  is  also  possible  to  train  ATR  algorithms  on  specific  image  features  and  to  use  the  comparative  performance  in  real 
and  synthetic  images  as  a  measure  of  similarity.  Work  in  this  area  has  just  begun. 
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ORGANIZATION 


4.  APPLICATIONS 


4.1.  ATR  training 


Doing  ATR  training  on  synthetically  generated  imagery  offers  several  obvious  advantages.  Besides  from  being  much 
cheaper  and  easier  than  recording  real  SAR  imagery,  it  is  easy  to  set  up  target  scenes  and  put  them  on  different 
backgrounds.  For  every  object  in  a  scene  the  type,  location  and  orientation  are  always  known  exactly,  which  is  a  pre¬ 
condition  for  the  learning  process.  It  is  very  convenient  to  have  the  possibility  of  automatically  generating  mask 
images,  because  in  many  cases  a  lot  of  imagery  is  required  to  get  an  ATR  algorithm  working  according  to  its 
specifications.  Manually  drawing  masks  for  large  amounts  of  imagery  drawing  can  be  very  time  consuming  and  error 
prone,  so  automatically  generating  masks  means  getting  better  ATR  results  in  less  time.  In  order  to  make  a  learning 
based  approach  rotation  independent,  it  needs  rotated  images  of  the  same  object  as  input.  These  are  also  very  easy  to 
obtain  with  synthetic  pictures.  Moreover  it  is  possible  to  simulate  many  different  sensor  types  using  a  single  tool  as 
long  as  the  sensor's  properties  are  known. 

4*  4-  **  **  V-  |  * 
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Figure  17  Automatic  generation  of  training  imagery  and  masks 


Figure  17  shows  a  training  scene  with  an  object  rotated  in  steps  of  5  degrees.  The  arrows  indicate  its  directions  (left 
image)  The  synthetic  SAR  image  that  was  generated  from  these  data  and  the  corresponding  mask  for  ATR  training  are 
shown  in  the  middle  and  right  image.  After  a  training  scene  has  been  set  up,  the  mask  image  is  generated  automatically 
together  with  the  SAR  image.  This  can  be  done  for  any  scene  by  activating  a  single  checkbutton  in  the  GUI.  Using 
additional  scripts  it  is  easy  to  save  scenes  for  repeated  use  (e.g.  with  different  targets  placed  in  them)  or  to  rotate 
scenes.  The  result  of  rotating  a  whole  scene  is  illustrated  in  figure  18.  Note  that  the  shadows  always  point  in  the  same 
direction  -  away  from  the  sensor  which  can  be  imagined  to  be  located  above  the  scene. 


Figure  18  Automatic  generation  of  training  imagery  and  masks 

4. 2. Integration  with  flight  simulator 

The  flight  simulator  at  EADS  is  used  for  the  ground  training  of  future  Eurofighter  Typhoon  pilots.  The  Simulation  and 
Scenario  Host  contains  a  simulated  aircraft  and  avionic  model  of  the  Eurofighter.  The  Aircraft  Simulation  comprises 
models  for  all  flight  critical  systems  as  well  as  for  the  environment.  Besides  the  human-piloted  aircraft,  other  aircraft 
(friendly  and  enemy  forces)  are  simulated.  A  graphical  user  interface  allows  easy  start-up  and  operation.  The 
Simulation  and  Scenario  Host  feeds  the  simulated  aircraft  and  avionic  model  data  to  the  cockpit’s  displays. 

The  cockpit  displays  are  driven  by  PCs  running  the  Linux  OS.  A  one  channel  projection  visual  system  shows  the 
outside  view  of  the  simulated  aircraft.  Overlayed  to  this  outside  is  a  HUD  projection.  A  sound  system  handles  audible 
warnings  and  sound  effects  (  engines,  gun,  etc. ). 
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Besides  the  many  computers  that  make  up  the  cockpit  environment  and  the  flight  simulation  there  are  some  more 
computers  with  very  specific  tasks.  The  collaboration  of  these  computers  is  illustrated  in  figure  19. 

There  is  a  mission  planning  computer  which  plans  flight  routes  according  to  the  special  geometry  of  long  range  SAR 
imaging.  One  of  the  more  serious  problems  there  is  area  shadowing  which  is  very  prominent  because  of  flat  incident 
angles.  The  waypoints  that  have  been  planned  by  the  mission  planner  are  then  transferred  to  the  central  simulation 
control  computer  which  distributes  the  data  to  the  cockpit  for  visualisation.  When  recording  waypoints  are  reached  the 
SAR  simulator  is  triggered  to  calculate  an  image.  The  image  is  then  transferred  to  the  avionics  development  computer 
where  the  ATR  algorithms  process  it.  The  result  is  returned  to  the  simulation  control  computer  which  relays  it  to  the 
cockpit  for  visualisation. 
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Figure  19  Components  of  flight  simulation 


Figure  20  Screenshot  of  ground  attack  display 


An  example  for  visualisation  of  the  SAR  image  together  with  classification  results  is  shown  in  figure  20.  The  SAR 
image  is  contained  within  the  blue  frame.  Symbols  overlayed  in  red  mark  target  positions  and  their  types. 


5.  USING  EXTERNAL  DATABASES 


For  the  application  of  a  SAR  simulator  in  mission  related  environment,  it  is  relevant  to  have  good  terrain  data  of  the 
scenario  in  question.  This  might  not  be  the  case  for  crisis  regions  like  Afghanistan.  Anyway,  it  is  still  possible  to 
extract  some  of  this  information  from  public  data.  The  following  example  illustrates  a  possible  approach  on  the 
example  of  the  Kabul  airport.  For  elevation  data  the  SRTM  (shuttle  radar  topograhy  mission)  data  was  used,  which  is 
available  on  the  internet  for  the  whole  surface  of  the  earth  at  a  3  arc  second  resolution.  The  cultural  data  were  extracted 
from  satellite  imagery  in  the  visible  spectrum.  This  can  be  done  manually  as  in  our  case  or  automatically  using  texture 
classification  algorithms. 


Figure  21  Intermediate  steps  in  the  creation  of  SAR  imagery  from  alternative  data,  from  left  to  right:  SRTM  tile  (a), 
extracted  height  map  for  Kabul  airport  (b),  satellite  image  (c)  and  extracted  cultural  data  (d) 
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Figure  21a  shows  the  SRTM  tile  for  a  latitude  of  34  and  longitude  of 
69  degrees  that  contains  both  Kabul  and  its  airport.  A  map  (not  shown) 
was  used  to  determine  the  coordinates  for  the  airport  and  to  do  the 
extraction  of  the  corresponding  height  data  (21b).  A  satellite  image 
(21c)  was  used  to  extract  feature  information  by  overlaying  the  image 
with  a  transparent  layer  and  painting  homogenuous  areas  in  the  same 
color.  The  resulting  image  is  shown  in  figure  22.  The  image  was  created 
using  DoSAR  parameters.  It  spans  an  area  of  about  2400x2400m  and 
is  simulated  in  X-Band.  The  pixel  resolution  is  0.57m,  the  elevation 
angle  is  57°  and  the  height  of  the  sensor  platform  is  8km.  Forested  areas 
were  processed  using  the  height  map  post-processing  introduced  in 
chapter  2.4  and  a  couple  of  airplane  models  were  placed  in  the  scene  for 
better  realism. 


6.  CONCLUSIONS  AND  FURTHER  WORK 


Figure  22  Artificial  SAR  image  of 
Kabul  airport 


A  system  for  dynamically  generating  SAR  imagery  from  digital  terrain  databases  was  developed.  An  evaluation  of 
several  parameters  influencing  image  realism  was  done.  Approaches  for  improving  image  realism  and  database 
enrichment  were  presented.  The  usability  of  synthetic  imagery  for  ATR  training  and  for  the  integration  into  a  combat 
flight  simulator  were  illustrated.  A  lot  of  work  remains  to  be  done  concerning  the  validation  of  the  imagery.  Statistical 
and  ATR  based  approaches  are  to  be  considered. 

A  new  SAR  image  generator  is  currently  developed  that  includes  spotlight  mode,  IMU  deviations  simulation  and  squint 
angles.  The  currently  used  statistical  methods  for  simulation  of  SAR  typical  effects  will  be  replaced  by  precise 
computations  of  the  physical  effects. 


7.  REFERENCES 

[1]  Kibbe,  McDowell:  “ Man-machine  interface  with  simulated  automatic  target  recognition  systems‘\  AGARD, 

Avionics  Panel  Symposium,  Madrid,  Spain,  1992. 

[2]  Barbato:  integrating  voice  recognition  and  automatic  target  cueing  to  improve  aircrew-system  collaboration  for 
air-to-ground  attack^,  RTO  Systems  Concepts  and  Integration  (SCI)  Symposium,  Ottawa,  1998. 

[3]  Stahl,  Haisch,  Wolf:  “ Automatic  Target  Recognition  Flight  Prototype  for  Police  Helicopters “,  SPIE  Proceedings, 
2002. 

[4]  Fries:  , JHessung  und  Modellierung  der  Leistung  von  Kantenoperatoren‘\  Fraunhofer  Institute,  Karlsruhe, 2003. 


22-10 


RTO-MP-SET-096 


UNCLASSIFIED/UNLIMITED 


Military  Aircraft 


EADS 

DEFENCE 
&  SECURITY 


Generation  of  synthetic  SAR  imagery  for  ATR 
development 


MATRIX  2005 
Heiko  Seidel 


Military  Aircraft 


Motivation 


tendency: 

more  imaging  sensors 


E  H-T  V 

-  '  —  "  -  : 

low  avaiiaDiniy 
of  real  SAR 

. .  BHH _ 

— 

imagery 

MUfr-.'iM.:- ■? ::  DBH1  = 

development  of  SAR 
simulator 


EADS 


DEFENCE 
&  SECURITY 


need  for  training  imagery 


Military  Aircraft 


Overview 

*  Introduction  to  EADS  SAR  simulator 
^  Image  enhancement 

*  Validation 

*  Applications  of  artificial  SAR  imagery 

*  External  databases 


Outlook 


EADS 


DEFENCE 
&  SECURITY 


Military  Aircraft 


Simulation  components 


EADS 

DEFENCE 
&  SECURITY 


database 

access 

module 


sensor  and  processing 
parameters 


target  location 
information 


module  output  and 
thumbnail  image 


output  control  settings 


image 

generator 


Military  Aircraft 


Image  generator:  process  of  image  synthesis 


EADS 

DEFENCE 
&  SECURITY 


feature  map 


radiometric 

expectation 

values  CAD-Models 


synthetic  SAR  image 


Military  Aircraft 


Image  generator:  3d  models  and  level  of  detail 


EADS 

DEFENCE 
&  SECURITY 


models  coming  with  models  from 

SARVIEW  Internet 


self-modified 

models 


Scud-Launcher 
vertices  :  28187 
surfaces:  56278 


T-72 

vertices  :  38738 
surfaces:  66634 


PATRIOT  missile 
launcher  unit 
vertices  :  83872 
surfaces:  163512 


MLRS 

vertices  :  26062 
surfaces:  46068 


MLRS 

„ready  for  launch11 


AWACS  E3A  Sentry 
vertices:  4466 
surfaces:  8932 


Military  Aircraft 


Terrain  database 


*  area  calculation 

*  input  map  generation 

*  target  coordinates 


.1 


EADS 

DEFENCE 
&  SECURITY 


*  edit  target  database 

*  add  other  objects 


Military  Aircraft 


Comparison  with  other  databases 

DFAD  level  2  ATKIS 


*  ATKIS  =  amtliches  topographisch-kartographisches  Informations-System 

*  different  level  of  detail 

*  realism  of  SAR  imagery 


EADS 

DEFENCE 
&  SECURITY 


Military  Aircraft 


Database  extensions  :  power  poles 


^  DFAD:  only  dimensions  of 
man-made  objects 
*  two  possibilities: 


*  left:  landscape  modulation 
(height  and  surface  cover) 

^  right:  3d  model 
model  data  into  landscape  or  *  3d  model  yields  more  realistic 
•  extract  coordinates  and  use  simulation  result 
3d  model 


.-L 


EADS 

DEFENCE 
&  SECURITY 


real  SAR  image 

source:  EADS,  MiSAR,  resolution  0.5m 
in:  Planet  Aerospace,  1/2004 


Military  Aircraft 


Database  extensions  :  buildings 


scene  (DFAD) 


SAR  image 


EADS 

AX.,  DEFENCE 
&  SECURITY 


SAR  image  with  added 
3d  models 


■  *  *  B-Q 

1 

f 

Military  Aircraft 


Height  interpolation 


bilinear  interpolation  spline  interpolation 


EADS 


DEFENCE 
&  SECURITY 


depression  angle:  8° 
carrier  frequency:  9.6  GHz 
chirp  bandwidth:  200  MHz 
resolution  :  0.75  m 


Military  Aircraft 


Database  post-processing  (1) 


EADS 

DEFENCE 
&  SECURITY 


*  forest  height  modulation 

*  road  surface  raised 

*  water  surface  lowered 


Military  Aircraft 


EADS 

DEFENCE 
&  SECURITY 


Database  post-processing  (2) 


*  Improved  post-processing  algorithms 
for 

■*  urban  areas 

■*  fields  /  meadows  /  shrubs 

*  vehicle  tracks 


Military  Aircraft 


in  collaboration  with: 


Validation 


Fraunhofer  ii^itut 

Ir-^rtwn  -mi-inrl 
I>mnwpm1^h-ung 


EADS 

DEFENCE 
&  SECURITY 


tarqet  siqnatures 


T-72 

MStar 


simulated 


Military  Aircraft 


Sensor  Simulation  and  ATR  development 


EADS 

DEFENCE 
&  SECURITY 


•  definable  scenario 


•  definable  configuration  of  target  objects 

•  precise  ground  truth  knowledge 


?  Y  V  iJ  *  A 

' 

^  l  u  -v 
' 


'  -H! 

■  ■■Vlr.Jjpi?  Xij 


training  scene 


testscene  in  database  and  ground  tru 


Military  Aircraft 


The  flight  simulator  at  EADS 


*  environmental  and  aircraft  models 

*  computer  generated  threats 

*  outside  view  and  HUD  projection 

*  SAR  image  from  given  aircraft  position 
and  sensor  attitude 


EADS 

AX  DEFENCE 
&  SECURITY 


Bu  (I  Sep  111  200!  1  5:56  00  < 


.  ■  ■  ■  i  "i" 

.V  . 


:n< 


L-  .■  ■■  ■  ■■■  ^  ■ 


■ 


Fuel 


FREU 


Military  Aircraft 


.i 


Simulations  with  external  databases: 
example  -  simulating  crisis  areas 


map  of  Kabul 


corresponding  SRTM  patch  satellite  image 


extracted  height  map  extracted  feature  map 


EADS 

DEFENCE 
&  SECURITY 


Military  Aircraft 


parameters: 

DoSAR  /  height:  8  km 


SAR  image 


Military  Aircraft 


EADS 

DEFENCE 
i  SECURITY 

Summary 


*  System  for  online  generation  of  SAR  images  integrated 


*  Database  post-processing  algorithms  for  realism  improvement  developed 


j  Validation  approach  being  defined 


*  New  image  generator  being  developed 


*  Application  examples 


*  Use  of  alternative  terrain  data  demonstrated 


Military  Aircraft 


Outlook 


EADS 

DEFENCE 
&  SECURITY 


imi-ilri  \ji  | 

_ 


'E/J-rt-tLii  .via  [■  ipK  v  n.r  Mi  ld  dnr|  "  |  !9  c  aldn  ■  !  rVr.wUn  1 1 


"  “  ■  ■  ,  ’■  mmM  (  "  ,«,■  ■"jsj  a  a  a  *  j  a  ^  i  ^  "  g*»  u  •  a  ‘  U>  a  '  a  I  "  a  aJ  r»_"  I  ‘p.""  " 

■ ' Tit Lj  i  rh  |  :e  rt  14  rnls'l  1  nr  n  d-ri'th, ■  1 1  |  '.'■rLB-i  ■  ~L|  ■■  rhrq,  1 1 


F1RH9.  PlairHirm-liidHFJiiiilviil  RfliigHiDi/p|jl«dmay*'?im  Jrfimi 

r.^flaffSSwawite1. , r  O  v.-r. . 


ifnr>iifrmh^ 
qi  .■ 


Ai-  il  Hrili  ■  aaa_i  l  an- 

"□«  11^  ZH  □3?'  ~F?1  *Tg  TWi 

rYu.M  Ll4iAji.vV.lt 

—  .v;  ”  r  T  IT  ” 


w.  :  Tfl  ”i£T 

D-ilriD^hrAtvM^I 


Iiiknp  :  ||"IL|  iJ-irit  in  fH  :  |IL‘ 

i 


I  -  J 

I  &1..:MI=nL 

T|ll  BIB  ■  pa  1 1  | 


iimjuaii  -  ii !  |gy  -JW 


*  New  image  generator  PIRDIS  for  simulating: 

*  Squint  angles 

*  IMU  deviations 

*  more  RCS  functions 
■*  Enhanced  target/target  and  target/surroundings  interaction 


EADS 


DEFENCE 
&  SECURITY 


Questions 

??? 


Military  Aircraft 


EADS 

AX  DEFENCE 
&  SECURITY 


Additional  Material 


Military  Aircraft 


Control  script :  data  flow 


EADS 

AX  DEFENCE 
&  SECURITY 


parameters 


f 

commands 

commands 

data  (position, image) 

f;. 

ground  truth 

1 

SAR-View  /  SAR-Target  modules 


Landscape  and 
feature  processing 


Calculation  of 
radiometric 
expectation  values 


target  processing 


target  insertion 


SAR-processing 

I  ~ 

image  generation 


==LI 


parameter 

files 


